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EXAMINER'S ANSWER 



This is in response to the appeal brief filed 3 April 2006, appealing from the Office 
action mailed 5 August 2005. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
substantially correct, only the reference Montgomery et al. is listed at page 1 1 as "U.S. 
Patent No. 5,969,533", where it should properly be US #5,696,533. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

5,945,976 IWAMURA et al. 8-1999 

5,696,533 MONTGOMERY et al. 1 2-1 997 
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(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 
Claims 1-38 are rejected under 35 U.S.C. 103(a) as being unpatentable 

over Iwamura et al. ("Iwamura"; US #5,945,976) in view of Montgomery et al. 

("Montgomery"; US #5,696,533). 

As per independent claim 1's "graphical user interface" in which "a rendered 
image of at least one graphical object" appears "on a display device", Iwamura's 
GRAPHIC DATA PROCESSING SYSTEM produces scene data by which a three- 
dimensional scene image is generated and displayed (Abstract). Please note such 
displays as Iwamura's figs 14A, 14B, in which "graphical object" depictions appear in a 
"graphical user interface". This is a three dimensional image with which a user interacts 
with an indication cursor (fig 5A,col 7, lines 5-16). 

While some form of "pixel" graphics perse is necessarily involved in Iwamura, 
Iwamura does not extend to providing explicit teachings of "a color value stored for 
each pixel" along with "object identification data stored with each pixel covered by the 
rendered image", whereby this "identifies the graphical object located at the pixel", 
though Iwamura's indication cursor appears to call up an "object" through some 
procedure. Also, it should be kept in mind that Iwamura must produce the graphical 
image and provide for proper access with the indication cursor , meaning both graphical 
rendering and pick-handling of some form should be used. 

However, Montgomery's METHOD FOR SELECTING AN ITEM ON A 
GRAPHICS SCREEN is one in which an item buffer is used to assign a unique item 
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identifier to members of a graphics object list (Abstract). The explicit teachings of 
Montgomery in this regard appear in fig 2, where an item buf with item num entries is 
run in parallel with a frame buf containing color values. This means that for each pixel 
color indication, there is an item that corresponds to it, as in applicant's "color 
value"/" object identification data stored with each pixel". 

Thus, it would have been obvious to a person having ordinary skill in the art at 
the time of applicant's invention to use the item buffer technique of Montgomery to 
assign "identification data" to the scene image objects of Iwamura. The motivation is to 
provide the Iwamura user with a direct indexing to the identities of the contents of the 
scene image , whereby the indication cursor when pointed to an Iwamura object, will 
properly return "object identification" from the pre-stored Montgomery item buf contents 
at that pointed-to location. The environment envisioned by Montgomery can extend to 
Painting and Texturing on 3D Shapes (col 2, lines 6 - 1 1 ), so that a motivation is also 
seen in Montgomery for application of the item buffer to three-dimensional pixel 
graphics like Iwamura's. 

What results in the combination is that an Iwamura image, rendered to appear 
proper in the two-dimensional display, has a buffered index for handling picks like that 
of Montgomery, where selection anywhere on the screen will return the identifier of the 
object pointed to. 

Claim 2's "three-dimensional image comprising at least two graphical objects that 
are co-located" (see also claims 12, 23, 30) is seen in Iwamura, where one building 
appears behind another in fig 14A. Using the Montgomery item buffer only the "visible 
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graphical object" will obtain the value for that object in memory. A similar line of 
reasoning applies to claim 13. 

The "writing" (claims 3, 14, 25) and "reading" (claims 4, 15) of "frame buffer" data 
occurs in Montgomery (thus "describing how each graphical element is to be displayed" 
along with an "object identification value" in a parallel "frame buffer" organization (see 
claim 26)), and the use of a "cursor pointing to a particular coordinate" (claims 5, 16) is 
part of both Iwamura, with the indication cursor and Montgomery, with a pointer device . 
In the combination of references, "reading the object identification data" (claims 6, 10, 
17, 27, 31) takes place in Montgomery, to supply "object identification data" for 
Iwamura's scene . 

Claim 7's "exporting the object identification information" (see also claim 18) is 
seen in Iwamura, where "metadata information" (claims 8, 19, 28, 32) as appears in the 
region 1207 or 1303 (fig 15A) is provided. This data is made part of an "object- 
identification value indexed data structure" (claims 9, 20), when the Montgomery item 
buffer values are used to call such data from Iwamura's store. See also claim 21, in 
which "a metadata information display object" reads upon Iwamura's supplemental 
scene display. 

Independent claim 1 1's "rendering an image of a plurality of graphical objects" 
reads upon Iwamura, for reasons similar to those given above for claim 1 , and the use 
of "a color value" and "object identification data for each of the specified locations" is 
then suggested by the combination with Montgomery, where "locations" are indicated by 
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the item buffer designations for the pixels that are involved. A similar line of reasoning 
applies to independent claims 22, 33. 

Independent claim 24's "associating a unique object identification value with each 
graphical element" is suggested by Montgomery, when applied to the "graphical 
element" composition of Iwamura's scene . There, the "data structure" behind the 
supplemental displays will add metadata to the item num listings in Montgomery. A 
similar line of reasoning applies to independent claim 29. 

The "computerized system" in claim 34, with its "pixels" that have "a unique 
object identification (ID) value" reads upon Montgomery, as noted above. The 
"rendering process" that determines "visible surfaces", as also discussed above, is seen 
in Iwamura. Montgomery then provides a "pixel map" to accomplish "object 
identification" of those objects that are Iwamura-rendered. 

The use of "a pointer" in relation to "at least one graphical object" (independent 
claim 35) is found in Iwamura, and the "frame buffer" whose "first field" is "for holding 
display information" and whose "second field" is "for holding object identification 
information" is suggested by Montgomery. 

The "frame buffer" of independent claim 36 is largely suggested by the "pixel" 
"memory locations" of Montgomery, which can be used in combination with the display 
of "a unique software graphical object" as per Iwamura. 

Independent claim 37's "graphical user interface", in which "there is stored a 
respective color value for each pixel in the display device", along with "object 
identification data stored with each pixel" also reads upon the combination of Iwamura's 
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"three-dimensional graphical object rendering", when combined with Montgomery's 
parallel buffering of color and item information. 

Independent claim 38's "rendered image produced using a z-buffer technique" is 
suggested by the display of overlapping objects with hidden surface removal in 
Iwamura, especially when considered in view of the item buffering of Montgomery, 
where a frontmost item is indicated for each pixel. The comparison and retention of 
closest z-values is a part of any arrangement in which pixels must be shown with 
topmost priority in a "three-dimensional graphical" display. Indeed, Iwamura identically 
teaches at col 8, lines 53 - 65 that the ground object data can be obtained by a Z buffer 
method . Claim 38's use of "object identification data stored with each pixel" is then 
seen, as has been noted above, in Montgomery, whose returning of a visible item 
identification for each pixel would have a direct usefulness in the access of related data 
as is illustrated in Iwamura. 

(10) Response to Argument 

At page 12 of the brief, appellant argues that "none of the references discloses, 
suggests or even appreciates a graphical user interface that includes a rendered image 
of at least one graphical object, wherein the graphical object uses a number of pixels on 
a display device and a color value stored for each pixel in the display device ". However, 
a rendered image like Iwamura's will produce an array of varied "color value" entries for 
a "graphical object". Montgomery is then relied upon to show the identical use of a 
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parallel "color valueTobject identification data" buffering arrangement, of the kind that 
"uniquely identifies the graphical object located at" a "pixel", as in independent claim 1. 

Appellant s argues at page 13 that "the Office Action has misinterpreted the 
claim limitation 'a color value stored for each pixel in the display device.'", in 
relying upon the reference Montgomery, in which "there is a but a single color available 
for each object" (see page 12 of brief). But it is pointed out that even in the identical 
disclosure of solid-colored objects used in Montgomery, there is "a color value stored for 
each pixel in the display device". The Examiner is not permitted to "read in" as 
appellant would, "that the only interpretation feasible dictates a separate/respective 
color value for each pixel in a display device", since neither "separate" nor "respective" 
actually appears in the claims. 

But even if the claim is made to require "separate" color values stored for a 
"graphical object", the combination with Iwamura suggests that colors become variable 
throughout the content of a scene, where objects there are not rendered in single colors 
across their units. Iwamura's rendering is ideally-suited to carry forward into a pick- 
testing scheme like Montgomery's, where the pixel-by-pixel colors are then read into the 
color-maintaining portion of memory along with the parallel item buffer . 

Appellant's argument is thus answered at page 13, where "Applicants do point 
out that not every pixel in the Montgomery device is covered by an object, only objects 
within the display" — Iwamura produces a rendering of an entire scene, as bounded by a 
rectangular border. All points within such a display should be rendered and 
addressable by the pointing device. 
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At page 14, appellant, in questioning the motivation given by the Examiner for the 
combination of the references, argues that "the stated motivation is not found in any 
reference of record, or supported by any basis or theory having support in the 
references of record". However, the Examiner has attempted to show that pixel 
graphics selectable by a pointer as in Iwamura provide a motivation, as seen in that 
reference, for some procedure for identifying which objects in the image have been 
picked, something that is done with the item buffer of Montgomery. Also, the 
Montgomery disclosure, as has been noted in above, is envisioned as being useful with 
3D imagery, this being a motivation found within Montgomery to use the item buffer with 
rendered 3D images like Iwamura's. 

At page 14 appellant also questions the combination by arguing that "it is not 
apparent that the virtual reality simulator of Iwamura is remotely compatible with, or 
would benefit from, the item buffer described in Montgomery as compared to the 
present (z-buffering) system used by Iwamura". However, the three-dimensional 
technique of z-buffering, as is mentioned in Iwamura, is principally used for hidden 
surface removal and the presentation of proper priority. It is essentially an image 
rendering technique, and not a pick-handling technique, this being accomplished by a 
further mapping to an item buffer such as Montgomery's, so that the objects shown as 
foremost by z-buffering are then able to accept pointer selection by the user. 

Appellant continues on the subject of inapplicability of Iwamura and z-buffering at 
page 15, arguing that "there is absolutely no subsequent use for any item-buffer as the 
z-buffering process already renders each pixel observable from a given perspective on 
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a two-dimensional display." But while rendering "each pixel observable" might be "a 
fundamental purpose of z-buffering", there is still a need for an all-points-addressable 
screen to be able to respond to item selections via a cursor. Simply rendering the 
image does not complete the process of indexing relative to the positions entered by the 
pointer. For this, there is suggested in Iwamura a need for an item buffer like 
Montgomery's. 

Appellant argues at page 15 that in Montgomery, "only two-dimensional systems 
are described, and only two-dimensional objects, e.g., rectangles and triangles, are 
discussed." However, in producing an image like Iwamura's it is first required that a 
two-dimensional representation, such as can be handled by Montgomery, be produced. 
Indeed, Montgomery actually refers to 3D rendering in the background of the invention, 
and can be seen as useful in that particular area of graphical rendering, presentation 
and interaction. 

Appellant's argument at pages 15 - 16 that "nowhere does Montgomery provide 
for any change in viewing perspective" is not relevant to the manner in which the 
references are relied upon and combined. Iwamura has ample teachings of 3D 
rendering, of the kind where viewpoint varies. It is the combination of references that 
the Examiner relies upon, and appellant is not entitled to attack references one at a 
time, given the motivation for that combination that exists. This also answers the 
argument at page 16 that "the process shown in Montgomery might be perhaps closest 
to a technique known as 'z-sorting.', since it is merely the parallel use of frame buffering 
for colors and item buffering for object selection that is relied upon in Montgomery, and 
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this has direct usefulness in an image produced by standard z-buffering such as 
Iwamura's. 

The Examiner disagrees, then, with appellant's argument at page 17; that "One 
cannot functionally append an item buffer to a z-buffer - it makes no sense technically 
and serves no purpose". The z-buffering to render an image needs a further mapping to 
handle picks, and Montgomery has immediate usefulness with Iwamura. The Examiner 
is not attempting to "replace a z-buffer with an item buffer", but rather, to use them 
together. Contrary to appellant's arguments, "item buffers" are "suited to three 
dimensional rendering", since such "rendering" produces a two-dimensional image of 
the kind Montgomery can present. But rendering alone is only part of producing an 
interactive 3D image like Iwamura's; it only presents the pixel values for display. There 
must also be a mapping of screen positions to objects that will react to the pointing 
device, and for this, an object-segmenting, item buffer arrangement like Montgomery's 
is suggested to accomplish this function. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 
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For the above reasons, it is believed that the rejections should be sustained. 
Respectfully submitted, 



Raymond J. Bayerl 
Primary Examiner 
Art Unit 2173 
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